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(54) System for optimizing the display and rendering of digital images for digital mastering 



(57) A digital mastering system for processing dig- 
ital image signals originating from scanned motion pic- 
ture film includes a film scanner for scanning motion pic- 
ture film and generating a stream of digital image signals 
corresponding to motion picture images, a color grading 
channel that pre-visualizes a certain "look" that can be 
applied to the digital image signals and a digital render- 
ing channel that renders the digital image signals pre- 
visualized for a certain "look" into a digital master. The 
"look" includes the appearances produced by either a 
standard motion picture print film and/or adjustments 
applied to a standard appearance to obtain a creative 



effect. The color grading channel includes a parameter 
generator for storing parameters for a plurality of 
"looks", a digital color grading processor for applying 
one or more selected parameters to impart a pre-visu- 
alized "look" to the digital image signals and a display 
device capable of reproducing the color graded signals. 
The digital rendering channel includes a digital render- 
ing processor for applying one or more of the parame- 
ters selected by the parameter generator to the digital 
image signals In order to impart the rendered "look" into 
the digital image signals and a digital mastering device 
for producing a digilal master from the rendered digital 
image signals. 
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Description 

FIELD OF THE INVENTION 



[0001] The present invention relates generally to the 
field of digital image processing to emulate motion pic- 
ture film tone scale and color. More specifically, the in- 
vention relates to rendering an electronic image to em- 
ulate a film system reproduction, that is, to provide the 
appearance of an image captured and displayed with 
motion picture film. 

BACKGROUND OF THE INVENTION 

[0002] The digital era that is now upon us has the po- 
tential to change the way everyone operates in the mo- 
tion picture field. The traditional method for projecting 
motion picture film was relatively simple because there 
were nol very many options available lo be controlled. 
Basically, film has a certain appearance or "look" that 
can be altered by processing, lighting and laboratory 
methods. The digital age has brought to the motion pic- 
ture industry the ability to capture a motion picture print 
imago and structure it in tho form of data. Once the data 
has been captured, it can be manipulated practically to 
any degree that is desired. However the more variables 
one has to work with, the harder it is to maintain or re- 
construct the original intent. 

[0003] A very large amount of program material po- 
tentially useful for electronic conversion exists, or is cap- 
tured, in the form of photographic film. The type of equip- 
ment capable of producing digital signals from such film 
material is usually called a telecine scanner, or telecine 
for short. However, a motion picture film that, when pro- 
jected in a dark auditorium, appears to be of perfectly 
satisfactory quality, may prove disappointing when con- 
verted for electronic viewing by means of a telecine. For 
this reason, it is possible to apply very useful adjust- 
ments to the pictures electronically. These corrections 
are typically applied by a colorist working in a telecine 
lab, and the corrected electronic signals are written out 
to a digital intermediate, such as a hard drive or other 
type of digital memory. In the terminology of the industry 
the colorist "grades" the scanned film to an optimum 
"look" for film or video output based on a CRT monitor. 
This requires a lot of guesswork on the part of the color- 
ist when the data is committed to film. 
[0004] Today colorists in a digital mastering or tele- 
cine lab are somewhat limited in the viewing environ- 
ment available to them while adjusting the tone scale 
and color of a digital image originating from scanned 
motion picture film or other digital capture device in or- 
der to produce an acceptable digital master. Equipment 
such as waveform monitors and vectorscopes can mon- 
itor the vast adjustment capabilities resulting from tele- 
cine and/or secondary color correction yet the motion 
imagery is typically displayed during this color grading 
process using a video monitor with a color gamut that is 



EP 1 239 668 A1 



smaller than the scanned film. Additionally, the current 
process for injecting a desired "look" (such as that of a 
release print film) while color grading, typically relies on 
1 -dimensional hardware look up tables (1D LUTs) im- 
5 plemented within these limited gamut video devices and 
the subjective creative control of a colorist. (The 1 D as- 
pect of a LUT means that one output value is looked up 
for each input value.) 

[0005] In the prior art, a given "look" can be attained 
10 through various adjustments using "primary" or "sec- 
ondary" color correctors, such as da Vinci™ color cor- 
rector offered by da Vinci Systems inc., with or without 
a telecine machine. In the patent literature, U.S. Patent 
No. 5.255,083 describes a digital color correction sys- 
'5 tern and method for color correction in a film-to-video 
signal conversion system. U.S. Patent Nos. 5 140 414- 
5,374,954; 5,406,326; 5,457,491 and 5,687,011 (each 
issued to Mowry) represent a family of related post-pro- 
duction video technology that seeks lo arrive at an aes- 
20 thetically acceptable simulation of the appearance that 
images originated on different motion picture film stocks 
would embody aftertelecine "flying spot scanner" trans- 
fer to video from taped high definition video originated 
images. One component of this prior art technology 
25 deals with the conversion of the video-originated mate- 
rial through a LUT that is based on color temperature of 
the scene lighting, scene brightness and selected f-stop 
setting. The conversion values in the LUT are derived 
by filming color charts and grey scale charts, obtaining 
30 a digital representation of the film component responses 
of the charts from telecine transfer of the film to video- 
tape, and then charting the telecine-derived component 
responses against video originated images of the same 
charts under identical lighting conditions. The final sim- 
35 ulated video image is either recorded as a high definition 
signal, or converted to an NTSC signal and broadcast 
or displayed. 

[0006] In the latter two of the aforementioned Mowry 
patents, the digitized video signal may be sent to a film 
40 recorder, which reproduces the component-modified 
images onto a selected, reversal film stock. The film is 
chemically processed with a film processor and then op- 
tically projected, or scanned to video, digital video or 
other electronic media. If the film recording option is em- 
ployed, these patents specify that it is important that the 
telecine-derived LUT used in the component modifica- 
tion involves response data which compensates for the 
inherent color response of the film stock on which the 
images are being digitally recorded. 
so [0007] These systems may be able to come percep- 
tually close to achieving a given "look" through trial and 
error, but have a tendency to introduce noise or other 
artifacts into the imagery in the process and are time 
consuming. In addition, there are colors that cannot be 
55 easily attained using conventional 1 D look up tables ( 1 D 
LUTs) and 3x3 matrices that may result from non-linear 
channel interdependencies. 

[0008] What is needed is a straightforward robust 
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method for emulating known "looks" (e.g. release film 
print) in real time on motion imagery with an expanded 
color gamut as part of an overall system that is capable 
of providing otherwise unattainable colors, all the while 
complementing existing products that perform color cor* 
rection on motion imagery. 

SUMMARY OF THE INVENTION 



[0009] It is an object of the invention to provide a sys- 
tem that allows a "look 0 (e.g. , of a print film) to be applied 
to digital motion data in real-time and to pre-visualize 
that "look" using a display device : such as a digital pro- 
jector. 

[0010] It is a further objective of the invention to pro- 
vide a color grading system that allows optimal opera- 
tion within the limited color gamut of a display device, 
and as part of an overall system that includes a custom 
hardware image processor thai can achieve real lime 
performance and improve the overall process. 
[0011] It is a further objective of the invention to use 
high speed LUT technology as a component in a series 
of configurable processing modules that allow the sys- 
tem to provide the "look" and obtain the otherwise un- 
obtainable colors. 
[0012] It is a further object of the invention to use a 
digital projector with an expanded color gamut as part 
of an overall system that includes a custom hardware 
image processor (e.g., including a 3D LUT or multiple 
1 D LUTs and 3x3 matrices) in orderto improve this proc- 
ess. 

[0013] The present invention is directed to overcom- 
ing one or more of the problems set forth above. Briefly 
summarized, according to one aspect of the present in- 
vention, a digital mastering system for processing digital 
image signals originating from a motion image source, 
such as scanned motion picture film, includes a color 
grading channel and a digital rendering channel. The 
color grading channel pre-visualizes a "look 11 that can 
be applied to the digital image signals, wherein the 
"look" includes the appearances produced by either a 
standard motion image source, such as motion picture 
print film, and/or adjustments applied to a standard ap- 
pearance to obtain a creative effect. To do this, the color 
grading channel includes (a) a parameter generator for 
storing parameters for a plurality of "looks" correspond- 
ing to one or more standard motion image sources and 
creative effects, (b) a digital color grading processor for 
applying one or more selected parameters to impart a 
pre-visualized "look" to the digital image signals, where- 
in the digital color grading processor also applies one 
or more transforms to the digital imago signals that are 
specific to a particular display device, thereby producing 
color graded signals that include the pre-visualized 
"look" and the display specific transforms, and a (c) a 
display device that reproduces the color graded signals 
as modified by the display specific transforms. 
[0014] The digital rendering channel renders the dig- 



ital image signals pre-visualized for a certain "look' into 
a digital master, wherein the rendered "look" includes 
the appearances produced by a standard motion image 
source and/or adjustments applied to a standard ap- 
5 pearance to obtain a creative effect. To do this, the dig- 
ital rendering channel includes (a) a digital rendering 
processor for applying one or more of the parameters 
selected by the parameter generator to the digital image 
signals in order to impart the rendered "look" into the 
w digital image signals, thereby producing rendered digital 
image signals that include the rendered "look" but are 
devoid of display specific transforms and (b) a digital 
mastering device for producing a digital master from the 
rendered digital image signals. 
is [0015] The invention provides the capability for a 
coloristto have an optimal view of the displayed results 
of color grading operations using a digital projector in a 
post-production facility. The invention includes config- 
urable processing steps that are implemented in hard- 
20 ware as 1 or more 3-dimensional look up tables, multiple 
1 -dimensional look up tables, and multiple 3x3 matrices 
loaded into a re-configurable compute engine. This 
compute engine will process motion imagery input from 
a digital source in real-time and send the optimized out- 
25 put to a digital projector or digital data storage device. 



Advantageous Effect of the Invention 

[0016] This invention allows for a colorist to more ac- 
30 curately color correct a digitized film source through a 
system that provides an appearance (and possibly a 
colorimetric) match between the digitally projected dig- 
ital source imagery and the desired end product (e.g. a 
projected release film print). This is an improvement 
35 over reliance on a limited color gamut device such as a 
video monitor. Visualization of external color adjust- 
ments made in this "color grading" process is optimal. 
The display-only scenario of the system allows for the 
pre-visuaiization processing to be customized to incor- 
40 porate projector specific transformations to emulate the 
desired end product "look" on any given digital projector. 
[0017] The "digital master" that can be created in a 
rendering process will more accurately reflect colorists' 
or other interested stakeholders' intentions and expec- 
ts tations. The color and tone scale of the motion imagery 
products (e.g. release film prints) that can ultimately be 
produced from the digital master will thus be optimized. 
[0018] Various end products (e.g. digital cinema dis- 
tribution master or release film print) from the same dig- 
50 ital master can be optimally pre-visualized through sim- 
ple changes to this system's 3D look up table setup pa- 
rameters using the same digital projector to achieve a 
consistent desired "look" across all end products. 
[0019] Any number of experimental "looks" can be 
55 rendered into the source imagery if desired when the 
system is used in a rendering mode. This presents an 
opportunity for cost and time savings over existing tra- 
ditional methods for achieving a particular "look" (e.g. a 
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"look" resembling what might be achievable through tra- 
ditional film chemical lab processes) in the overall post- 
production process. 

[0020] These and other aspects, objects, features 
and advantages of the present invention will be more 
clearly understood and appreciated from a review of the 
following detailed description of the preferred embodi- 
ments and appended claims, and by reference to the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0021] 

Figure 1 is a block diagram of a digital mastering 
system according to the invention, involving a digital 
color grading channel which allows a colorist to pre- 
visualize a particular "look" and a digital rendering 
channel which allows the colorisl lo render thai 
"look" into the mastered data. 
Figure 2 is a block diagram of the processing stages 
involved in generating the particular "look" in each 
of the channels shown in Figure 1 , which in a color 
grading view-only scenario of the invention allows 
a colorist to pre-visualize the "look" and any exter- 
nally applied color and tone scale adjustments with- 
out actually rendering the "look" into the digital 
source data, and which in a digital rendering sce- 
nario of the invention allows the colorist to render 
creative "looks" into the data (i.e. into a digital mas- 
ter) devoid of projector specific transforms. 

DETAILED DESCRIPTION OF THE INVENTION 

[0022] Because video or digital image processing 
systems that emulate the "look" of motion picture film, 
particularly after telecine conversion, are well known, 
the present description will be directed in particular to 
attributes forming part of, or cooperating more directly 
with, a system and a method in accordance with the 
present invention. Attributes not specifically shown or 
described herein may be selected from those known in 
the art. In the following description, a preferred embod- 
iment of the present invention would ordinarily be imple- 
mented at least In part as hardware, although those 
skilled in the art will readily recognize that the equivalent 
of such hardware may also be provided in software. Giv- 
en the system as described according to the invention 
in the following materials, the design of the hardware 
and software not specifically shown, suggested or de- 
scribed herein that is useful for implementation of the 
invention is conventional and within the ordinary skill in 
such arts. 

[0023] If the invention is implemented as a computer 
program, the program may be stored in conventional 
computer readable storage medium, which may com- 
prise, for example; magnetic storage media such as a 
magnetic disk (such as a floppy disk or a hard drive) or 



magnetic tape; optical storage media such as an optical 
disc, optical tape, or machine readable bar code; solid 
state electronic storage devices such as random access 
memory (RAM), or read only memory (ROM); or any oth- 

s er physical device or medium employed to store a com- 
puter program. If the invention is implemented in hard- 
ware, it may be implemented as an integrated circuit, 
such as an application specific integrated circuit (ASIC), 
or by other means known in the art. 

w [0024] This invention encompasses the real-time 
color processing that would occur on the image data 
stream between the machine producing the image data 
stream, such as a telecine or a digital data storage de- 
vice, and a display device such as a digital projector in 

is a digital mastering or telecine suite, a film recorder or a 
digital master rendering device. 

[0025] The real-time color processing capabilities of 
this system would provide an optimal way of interpolat- 
ing colors caplured on film that are outside the color 

20 gamut of a given projector (or other targeted viewing de- 
vice) and allow each display device to be characterized 
individually. Additionally, a film-like "look" or other crea- 
tive "look" can be applied to the data stream (with or 
without rendering the "look" into the data) using this sys- 

25 tern in a "digital intermediate" environment. 

[0026] The invention is composed of a set of recon- 
figureable hardware processing elements that can be 
configured with data that manipulates motion imagery 
in a digital mastering environment. This data allows the 

30 operator to inject a given "look" on that data that can 
either be "rendered" to a storage device in a rendering 
mode, or digitally projected in a display-only mode. More 
specifically, this invention provides a system that: 



35 



40 



45 



Allows an optional display-only "look" to be applied 
to the data stream in real-time to pre-visualize the 
"look" and any externally applied color and tone 
scale adjustments without actually rendering the 
"look" into the digital source data. Furthermore, this 
system optimizes the source imagery for the color 
gamut of the digital projector displaying the output 
of this system. This is shown in a color grading pre- 
view channel 12 in Figure 1. 
Allows an optional creative "look" to be rendered in- 
to the data (i.e. into a digital master) devoid of pro- 
jector specific transforms. This is shown in a digital 
rendering channel 14 in Figure 1 . 



[0027] Referring to Figure 1 , a digital mastering sys- 
50 tern 1 0 includes a color grading preview channel 12 and 
a digital rendering channel 14. Both channels 12 and 14 
process digital motion picture signals from a motion pic- 
ture film scanner 20, such as the SpiritDataCine™ tele- 
cine offered by Philips. However, the data that can be 
55 operated on is essentially digital image data from any 
given digital motion image source, such as a digital cam- 
era, a digital computer <computer generated images, 
animations, etc.) or a digital storage device containing 
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a source of digital motion images. In practice, although 
the invention does not inherently limit the scope of the 
input format in any way ; the data needs to be provided 
in a format supported by the system. A typical scanner 
provides YCbCr signals in the color grading channel 1 2 
and RGB signals in the digital rendering channel, al- 
though RGB signals could be provided in both channels. 
In the color grading channel 14 : the YCbCr signals are 
modified in a "look" processor 22 to apply a certain 
°look° to the signals. In the context of this disclosure, 
"look" is meant to include without limitation the appear- 
ance produced by the tone scale and colorimetry of a 
standard digital image source, such as source imagery 
from a digital camera, computer generation or a motion 
picture print film, and the adjustments a director would 
apply to a standard appearance to obtain a creative ef- 
fect. For example, a standard "look* might be the tone 
scale and colorimetry generated by printing and project- 
ing Kodak Vision Print Film 2383™, and a creative ad- 
justment might be the effect obtained by applying a skip 
bleach process to that source, doing print exposure 
flashing, offsetting the balance position, adding or rec- 
reating the tone scale and colorimetry of a discontinued 
film product, or the like. 

[0028] The creative and standard film "look" parame- 
ters are input from a parameter generator 24, which al- 
lows a myriad of different "looks" and creative effects to 
be imparted to the digital signals. The parameters are 
either precalculated values, e.g., a set of LUT entries, 
or could be generated contemporaneously in response 
to a colorist's input. In the preferred embodiment, the 
"look" processor 22 is a reconf igurable compute engine, 
and parameter generator 24 can provide a plurality of 
pre-computed parameters for various "looks" and crea- 
tive effects. In addition, a secondary color corrector 26, 
such as the da Vinci™ color corrector, may be utilized 
to provide additional color correction either as part of 
the scanning process (as shown) or elsewhere in the 
process. In fact, the invention complements the fine tun- 
ing color and tone scale adjustments made with the te4- 
ecine and/or secondary color correctors while color, 
grading. In addition, the invention provides a better start- 
ing point for such corrections. 

[0029] The processed digital signals are then viewed 
on a digital display device 30, which is preferably a wid- 
er-gamut device such as a digital projector. Projectors 
suitable for this purpose are generally available, and in- 
clude the DLP Projector offered by Texas Instruments 
Corporation and the G4000 Projector offered by JVC 
Corporation. Because certain signal transformations 
are required by specific display devices, the "look" proc- 
essor 22 also applies display specific transforms to the 
digital signals. As with the "look" parameters, the pa- 
rameter generator 24 may allow for transforms appro- 
priate for a plurality of display devices, A particular dis- 
play specific transform that is of interest is an algorithm 
for re-mapping any output values that are out of the 
gamut of the display device such that such values would 



be within the display device gamut. In one aspect of the 
invention, use of a digital projector provides a wider 
gamut than a conventional CRT display device and can 
display a broader range of colors, albeit some colors 
5 provided by the "look" processor 22 that are out of gamut 
would be re-mapped into the gamut of the digital projec- 
tor by means of the appropriate display specific trans- 
form. In this manner, an optimal display environment is 
provided. 

w [0030] In a film recording path of the digital rendering 
channel 1 4, the RGB signals are modified and adjusted 
in a "look" processor 32, which operates similarly to the 
"look" processor 22 except that only the creative effects 
are applied (the standard film "look" is inherently impart- 

15 ed by the film processing and usage). The specific pa- 
rameters generated by the parameter generator 24 for 
the color grading channel can also be input on a line 34 
to the "look" processor 32. The processed signals are 
either recorded on an intermediate master 36 or applied 

20 directly to a film recorder 38, which converts the proc- 
essed digital signals into optical signals that expose a 
release film print master 40. Distribution prints are then 
obtained from the master 40. 

[0031] In a digital projection path of the digital render- 
25 ing channel 14, the RGB signals are modified and ad- 
justed in a "look" processor 42, which operates similarly 
to the "look" processor 22 except that display specific 
transforms are not imparted to the signals. The specific 
parameters generated by the parameter generator 24 

30 for the color grading channel can also be input on a line 
44 to the "look" processor 42. The processed digital sig- 
nals are then recorded as a digital master 46, such as 
magnetic tape, a hard drive or a CD/DV-ROM. (Alterna- 
tively, the display data may be directly transmitted to the 

35 location of the digital projector, e.g., by a satellite trans- 
mission linkage.) Many digital copies 48 are then copied 
from the digital master 40 and distributed to a plurality 
of cinema locations 50, where they are modified in a 
processor 52 with projector specific transforms and pro- 

40 jected for a viewing audience with a digital projector 54. 
[0032] It should be understood that several variations 
of the digital projection path of the digital rendering 
channel 14 may be practiced. In one variation, the pro- 
jector specific transforms may be Implemented by the 

45 M iook" processor 42 if the requirement of the specific pro- 
jector 54 is known during the rendering stage. In another 
variation, all of the "look" and creative parameters may 
be stored, e.g., on the digital master 46, and distributed 
via the copies 48 to the cinema locations, where the 

50 processor 52 would have the capability of processing 
the "look" and creative parameters in place of the "look" 
processor 42 (in which case, the "look" processor 42 
could be inactivated or eliminated). 
[0033] Referring to Figure 2, the processing stages 

55 implemented by the "look" processors 22, 32 and 42 in 
the in the color grading channel and the digital rendering 
channel are presented side-by-side to emphasize simi- 
larities and identify differences. Recalling that the mo- 
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tion picture film scanner 20 may provide YCbCr data or 
RGB printing densities, if YCbCr data conforming to an 
input standard such as SMPTE 292M is provided to the 
color grading channel 12, a sampling stage 100 pro- 
vides up-sampling of source component pixel informa- 
tion from the input standard to the sampling ratios of the 
color space expected by the subsequent 3D LUT 
processing step (e.g. from a4:2:2 YCbCrsampling ratio 
to a 4:4:4 YCbCrsampling ratio). In the digital rendering 
channel 14, however, this step would be unnecessary 
as the source imagery should already be in a fully sam- 
pled (4:4:4) RGB color space in a format such as 
SMPTE DPX format (e.g. as film "printing densities"). 
(Staying in an RGB color space is considered one in- 
dustry standard way of maintaining color fidelity when 
color processing motion imagery.) 
[0034] It may then be necessary to provide a color 
space conversion from the source color space to an in- 
ternal color space of the system. For instance, in a color 
space transformation stage 1 02, the source color space 
is converted (e.g., by a 3x4 matrix conversion) to the 
color space expected by the subsequent 3D LUT 
processing step (e.g. from a 4:4:4 YCbCr color space to 
a 4:4:4 RGB printing density color space). However, in 
the digital rendering channel 14, this step would be un- 
necessary as the source imagery should already be in 
an RGB printing density color space. 
[0035] Next, the digital signals in each channel are 
processed in a sequence of look up tables (LUTs). First, 
bit depth scaling of the digital image signal source from 
1 0 bits/channel to 1 2 bits/channel is performed in a scal- 
ing step, using a 1D LUTs 104 in each channel. This 
LUT is also used to implement a function to allow for 
non-linear interpolation between nodes of a 3D look up 
table that otherwise would be linear as implemented in 
the hardware due to the inherent fixed spacing between 
the nodes of the 3D LUT in hardware. Additionally, any 
traditional 1D LUT processing (e.g. gamma adjustment) 
can be implemented at this point. The distinguishing el- 
ement within the workflow of the invention is the appli- 
cation of a re-configurable 3D look up table ASIC (in 
combination with the other matrices and 1D look up ta- 
bles implemented in FPGA hardware) in a group of 3D 
LUTs 106 f 108 and 110. A 3D LUT provides a known 
technique of interpolation over a regular grid of points, 
or nodes, in three dimensions, where the input is a triad 
(e.g., RGB) of values and the output is a triad of proc- 
essed values. Color processing is obtained by using 12 
bit image data through the 3D LUT ASIC, enabling a 
colorist to add a given "look" to the data stream (e.g. a 
specific release print film "look" or othercreative "look"). 
[0036] Commonly-assigned, copending U.S. Serial 
No. 09/797,891, "Method of Digital Processing for Dig- 
ital Cinema Projection of Tone Scale and Color", filed on 
even date herewith in the names of M.J. Bogdanowicz, 
C.V. Hume and CP. Hagmaier, which is incorporated 
herein by reference, describes in detail the chain of im- 
age processing that is implemented by means of the 3D 



LUTs 106,108 and 110. Briefly summari7ed, that disclo- 
sure describes a method for processing a sequence of 
digital image signals in order to match the appearance 
of a particular display medium, such as a motion picture 
5 print film, including the steps of (a) converting the digital 
image signals into signal values in an analytical dye 
amount space where the signal values are proportional 
to a dye set corresponding to the particular medium; and 
(b) converting the signal values in analytical dye amount 
10 space into output values suitable for a display device, 
such as a digital projector. Moreover the signal values 
in analytical dye amount space may be corrected for 
viewing conditions, such as printing flare, gamma, con- 
trast ratio or neutrality. There are other aspects to the 

15 disclosure, including the steps of: converting the signal 
values in analytical dye amount space into a perceptual 
color space based upon a chosen illuminant for display 
of the image signals; and converting the signals in the 
perceplual color space into output values suitable for a 

20 display device. 

[0037] Copending U.S. Serial No. 09/797,891 pro- 
vides a fast and reliable process for matching a digital 
source to a motion picture print film release. It therefore 
greatly minimizes the work involved by a colorist to ad- 

25 just the flesh tone and colorimetry of image sequences, 
therefore saving time and enabling the use of digital cin- 
ema projection. The method employed transforms the 
digital image sequence by simulating a "film model" for 
characteri7ing how film dyes appear when viewed with 

30 light. The tone scale and tone scale manipulations are 
operated on in a channel independent space, preferably 
a space referred to as an Analytical Dye Amount space. 
Analytical dye amounts are linear proportions of the in- 
dividual cyan, magenta and yellow dye spectrophotom- 

35 etry. The target "look" for a film based emulation is con- 
structed from the appearance of the image made from 
cyan, magenta and yellow dyes. The choice of Analyti- 
cal Dye Amount space allows the balancing (shifting) of 
the tone scale curves without imparting errors due to 

40 cross talk between the channels. The dye amounts are 
then converted and mapped to the target digital projec- 
tor. Consequently, the 3D LUTs 106, 108 and 110 use 
table values based on the analytical densities that are 
used to transform a digital image sequence to the target 

45 film model. The parameters for the plurality of film 
"looks" and special effects, as well as the display spe- 
cific transformations, would be stored with the parame- 
ter generator 24 as table values for the 3D LUTs 106, 
108 and 110. 

so [0038] Basically, as described in the commonly-as- 
signed, copending U.S. Serial No.. ..(Docket 82172), the 
image processing chain represented in the separate 
LUTs 106, 108 and 110 can each be represented as a 
mathematical model. For each RGB input set, there is 

55 a potential unique set of output RGB values defined by 
the mathematical model. The modeling processing is in- 
dependent of implementation. Forthis work, a fast meth- 
od of computation is needed to perform the conversion. 
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A tetrahedrai 3D interpolation method is used to simplify 
the amount of modeling calculations necessary to be 
made. Detailed information regarding a tetrahedrai 3D 
interpolation can be found in commonly-assigned U.S. 
Patent No. 4 t 992 : 861 , which is issued to John D'Errico 
and incorporated herein by reference. For this reason, 
the key processing steps are implemented in the recon- 
figurable "look" processors 22 : 32 and 42 as program- 
mable 3D LUT ASIC chips, or other high speed hard- 
ware, that preferably implement the 3D LUT processing 
described in the D'Errico patent. This is important since 
the system is intended to operate in real time. 
[0039] The bit depth scaled signals are processed in 
the LUTs 1 06, 1 08 and 1 1 0 to provide the tone scale and 
color modeling associated with the creative 'look" and ^ 
the standard "look", and with the specific transforms re- 
quired by the digital projector. The standard "look" is 
preferably that of a motion picture film projection sys- 
tem, while the creative "look" covers the modifications 
and adjustments applied by the director to the standard 20 
"took" to achieve the particular creative expression de- 
sired. Basically, in the color grading channel 12 the LUT 
1 06 applies the creative "look" and the standard "look", 
while the LUT 110 applies the display specific trans- 
forms. Alternatively, all three adjustments could be done 2s 
in one LUT, or the adjustments could be separated into 
three separate LUTs. 

[0040] The color re-mapping method employed in the 
aforementioned commonly-assigned, copending U.S. 
Serial No. 09/797,891 filed 02 March 2001 is an exam- 3( 
pie of a particularly useful display specific transform in 
the context of this invention. That re-mapping method 
employs the following methodology to process a color 
set that is out of gamut. First, the maximum color gamut 
of the display device is determined in a device-inde- a 
pendent perceptual color space. The chosen perceptual 
color space is CIE LAB, which is well understood by 
those of skill in these arts, but any other perceptual color 
space, such as XYZ tristimulus or CIE LUV space, would 
work as well. In CIE LAB space, the maximum color 4 
gamut is determined by setting the maximum saturation 
at each of the hue angles and calculating the Lightness 
value (L) at each point. The calculated Lightness value 
is used as a starting position for re-locating a color that 
is out of gamut. Then the hue angle is obtained for each < 
out of gamut color. This hue angle is used to lookup the 
starting Lightness value for the working color. 
[0041] This gamut re-mapping algorithm attempts to 
conserve the hue angle of the target color. Perceptually, 
changes in hue angle are more apparent than changes * 
in lightness or saturation. Since the hue angle is main- 
tained, the lightness and saturation must be altered to 
bring the target color into the display device color space. 
If the target color is lighter than the maximum saturation 
lightness, then the target color is darkened and desatu- 
rated to intersect with the display device color space. If 
the target color is darker than the maximum saturation 
lightness, then the target color is lightened and desatu- 



rated to intersect with the display device color space. 
While the foregoing gamut mapping technique is pre- 
ferred, other gamut mapping techniques may be used 
as display specific transforms to assign valid RGB code 
values to otherwise undisplayable colors. 
[0042] In the rendering channel 1 4, one LUT 1 06 pro- 
vides a creative "look" plus a standard film "look" of the 
type normally imparted by a motion picture print film. 
The other LUT 108 provides only the creative "look". 
Since the film produced by the film recorder 38 inher- 
ently imparts the film "look" of the particular motion pic- 
ture print film being used, there is no need in the LUT 
1 08 to provide a standard "look". Only the creative "look" 
desired by the director needs to be provided. Converse- 
ly, in the digitally mastered work that is to be distributed 
to exhibitors for digital projection, it is necessary to pro- 
vide both the creative "look" desired by the director and 
the standard "look" imparted by a particular motion pic- 
lure print film, since film is not being used in projection. 
However, the display specific transforms can be omitted 
at this point, since the projection site can apply these 
transforms. 

[0043] Projector specific, and/or creative or film "look" 
parameters, e.g., LUT entries, can be uploaded to any 
of the 3D LUTs from the parameter generator 24, there- 
by providing a mechanism for providing a myriad of 
"looks", creative effects and projector specific trans- 
forms. 

[0044] In each channel, a further scaling step using a 
1D LUT 112 provides bit depth scaling from the 12 bits/ 
channel output of the 3D LUT processing steps to 10 
bits/channel, plus any traditional (e.g. gamma adjust- 
ment) 1D LUT processing that would need to be done 
at this point. In the rendering channel 14, the data 
streams from the separate 1 D LUTs 1 1 2 would be written 
to some physical medium (e.g. as a digital master), such 
as the digital masters 36 and 46 (Figure 1 ), respectively. 
The mastering device for producing such digital masters 
may simply be the writing capability of a hard drive, a 

• writer for CD/DV ROMs, a magnetic tape recorder, or 
the like, and the resulting master may be no more than 
a section of the hard drive, etc. The digitally mastered 
output could now be used 1) to write image data onto 
film using a film recorder 38 (e.g. for a release print mas- 

> ter) and/or 2) to create a digital cinema distribution mas- 
ter 46 for playback/projection using the digital projector 
54. 

[0045] In the color grading channel 1 2, either the RGB 
data is directly provided to the digital display 30 or con- 

0 verted into the color space and sampling ratio expected 
by the display device, if not 4:4:4 RGB. In the latter case, 
a color space transformation stage 114 (using a 3x4 ma- 
trix conversion) converts from the color space produced 
by the 3D LUTs 1 06 and 1 1 0 to the output color space 

5 for the destination device (e.g. from 4:4:4 RGB projector 
display primaries to 4:4:4 YCbCr as per ITU 709 and 
SMPTE 292M). Moreover, it may be necessary at this 
point to provide down-sampling of source component 
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pixel information in a sampling stage 116 to an output 
standard such as SMPTE 292M, as would be expected 
by certain destination display devices (e.g. from a 4:4: 
4 YCbCr sampling ratio to a 4:2:2 YCbCr sampling ra- 
tio). 5 
[0046] Depending on the mode of operation of the 
system (display only, or digital rendering for either pro- 
jection or film recording), and format of the source or 
destination image data, some of the processing steps, 
as explained above, become optional. The key process- 10 
ing step of the digital mastering system is the tone scale 
and color modeling applications, the creative effects and 
the display specific transforms that are implemented in 
the reconfigurable 3D LUTs 106, 108 and 110 as high 
speed hardware, since the system is intended to oper- 15 
ate in real time. 

[0047] This invention allows for a colorist to more ac- 
curately color correct a digitized film source through a 
syslem lhal provides an appearance (and possibly a 
colorimetric) match between the digitally projected dig- 20 
ital source imagery (on the display device 30) and the 
desired end product (e.g. a projected release film print 
40 or digital master 46). This is preferably accomplished 
on a digital projector, which is an improvement over re- 
liance on a limited color gamut device such as a video 25 
monitor. Visualization of external color adjustments 
made in this "color grading" process is optimal. In addi- 
tion, the display-only scenario of the system allows for 
the pre-visuali7ation processing to be customized to in- 
corporate projector specific transformations in the LUT 30 
110 to emulate the desired end product "look" on any 
given digital projector. 

[0048] The "digital master" that can be created in the 
digital rendering channel 14 will more accurately reflect 
colorists* or other interested stakeholders' intentions 35 
and expectations. The color and tone scale of the motion 
imagery products (e.g. release film prints 40) that can 
ultimately be produced from the digital master 36 will 
thus be optimized. Various end products (e.g. digital cin- 
ema distribution master 46 or release film print 40) from 40 
the same digital master can be optimally pre-visualized 
through simple changes to this system's 3D look up ta- 
ble setup parameters using the same digital projector 
30 to achieve a consistent desired "look" across all end 
products. 45 



Claims 



A digital mastering system for processing digital im- 
age signals originating from a motion image source, 
said system comprising: 

a color grading channel for pre-visualizing a 
"look" that can be applied to the digital image 
signals, wherein the "look" includes at least one 
of appearances produced by either a standard 
motion image source or adjustments applied to 
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a standard appearance to obtain a creative ef- 
fect, said color grading channel comprising 

(a) a parameter generator for storing pa- 
rameters for a plurality of "looks" corre- 
sponding to one or more standard image 
sources or creative effects, 

(b) a digital color grading processor for ap- 
plying one or more selected parameters to 
impart a pre-visualized "look" to the digital 
image signals, thereby producing color 
graded signals that include the pre-visual- 
ized "look", wherein the digital color grad- 
ing processor also applies transforms to 
the digital image signals that are specific to 
a particular display device, thereby produc- 
ing color graded signals that include the 
pre-visualized "look" and the display spe- 
cific transforms, 

(c) a display device capable of reproducing 
the color graded signals as modified by the 
display specific transforms; and 

a digital rendering channel for rendering the 
digital image signals pre-visualized for a certain 
"look" into a digital master, wherein the certain 
"look" includes at least one of said appearanc- 
es produced by either a standard motion image 
source or adjustments applied to a standard ap- 
pearance to obtain a creative effect, said digital 
rendering channel comprising 

(a) a digital rendering processor for apply- 
ing one or more of the parameters selected 
by the parameter generator for the pre-vis- 
ualized "look" in order to impart the ren- 
dered "look" into the digital image signals, 
thereby producing rendered digital image 
signals that include the rendered "look" but 
are devoid of display specific transforms, 
and 

(b) a digital mastering device for producing 
a digital master from the rendered digital 
image signals. 

The digital mastering system as claimed in claim 1 
wherein the motion image source is a scanned mo- 
tion picture film. 

The digital mastering system as claimed in claim 1 
wherein the transforms that are specific to a partic- 
ular display device include an algorithm for rc-map- 
ping out of gamut color graded signals into the gam- 
ut of the color space of the display device. 

The digital mastering system as claimed in claim 1 
wherein the digital color grading processor applies 
the "look" associated with both the appearance pro- 
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duced by a standard motion image source and ad- 
justments applied to the standard appearance to 
obtain a creative effect, and the color graded sig- 
nals produced therefrom are used to drive a digital 
projector to display the color graded digital image 5 
signals. 

5. The digital mastering system as claimed in claim 1 
wherein the digital rendering processor applies the 
"look" associated with adjustments applied to a 10 
standard appearance to obtain a creative effect, 
and the digital master produced therefrom is used 

to drive a film recorder to record the rendered digital 
image signals on a motion picture film, wherein the 
film inherently imparts the appearance produced by '5 
a standard motion picture print film. 

6. The digital mastering system as claimed in claim 1 
wherein the digital rendering processor applies the 
"look" associated with both the appearance pro- 20 
duced by a standard motion image source and ad- 
justments applied to the standard appearance to 
obtain a creative effect, and the digital master pro- 
duced therefrom is used to drive a digital projector 

to display the rendered digital image signals. 25 

7. The digital mastering system as claimed in claim 1 
further comprising a film scanner for scanning a mo- 
tion image source comprising motion picture film 
and generating a stream of digital image signals 30 
corresponding to motion picture images. 
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